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SPECIALLY FORMATTED PAPER BASED 
APPLICATIONS OF A MOBILE PHONE 

BACKGROUND OF THE INVENTION 

Technical Field of the Invention 

The present invention relates in general to the 
communications field, and in particular to an interaction of an 
electronic reading device with an address pattern. 

Description of Related Art 

Numerous man-machine-interfaces (MMIs) exist for accepting 
user input and controlling user interaction with desktop and 
portable computers, personal digital assistance (PDAs) , mobile 
phones, and other types of electronic devices. For example, a 
keyboard can be used to accept typed input and other types of 
commands, a mouse or a track-ball can be used to provide 
relative motion input as well as various types of point-and- 
click selections, a keypad can be used to provide input of 
numerical data and functional commands, navigational keys, or 
scrolling wheels can be used for scrolling lists or otherwise 
repositioning a cursor, voice recognition can be used for 
performing certain operations, and various types of touchpads 
or touchscreens can be used to provide absolute positional 
coordinate inputs. Each type of mechanism for accepting input 
and for supporting user interaction has benefits and 
disadvantages in terms of size, convenience, flexibility, 
responsiveness, and easy of use. Generally, the selection of 
a particular type of MM I is dependent upon the function of the 
application and the degree and type of interaction require 

With the ever expanding capabilities and availability of 
applications both on the Internet and the area of wireless 
technology, there continues to be a need to develop and provide 
new MMIs for accepting input and interacting with users. In 
particular, some of the existing technologies suffer from 
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drawbacks or limitations, such as size and flexibility, that 
make them impractical and/or inconvenient to use in some 
situations. For example, MMIs that are typically used with 
mobile phones are not very fast and are not particularly user 
friendly, especially for sending long SMS messages or changing 
a lot of data in the device. Typically, long messages are 
entered using T9 (i.e., word predictive input), a keyboard 
attached to the phone (i.e., a chatboard) , a pen on a touch 
sensitive screen, or a numeric keypad where each key has to be 
pressed more than once to enter certain letters. Each of these 
alternatives suffer from a lack of speed and/or user- 
friendliness. By expanding the range of possible MMIs for 
supporting user interaction, application developers and end- 
users can have greater flexibility in the selection of input 
devices. Preferably, any such new mechanisms will provide 
increased flexibility and will maximize user convenience. In 
addition, the development of new mechanisms for interacting 
with users can expand the realm of potential applications. 

For example, while a keyboard typically provides a great 
deal of flexibility, particularly when it is used in connection 
with a mouse, a touchscreen, or other navigational device, its 
size makes it inconvenient in many cases, especially in the 
wireless context. 
' SUMMARY OF THE INVENTION 

The present invention comprises a method and system for 
data entry using an electronic pen. At least one position on 
a specially formatted surface is detected using an optical 
detector of the electronic pen that detects a portion of an 
address pattern of the specially formatted surface. Data 
relating to the at least one detected position is transmitted 
via a wireless interface to a separate electronic device. The 
data received by the electronic device is then used to perform 
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an operation corresponding to an area of the predefined address 
pattern that contains the at least one detected position. 

In accordance with a preferred embodiment of the 
invention, the separate electronic device is a mobile phone, 
PDA, personal or laptop computer, or other electronic device. 
The position on the address pattern at which the electronic pen 
is detected can be used to perform one of a variety of 
different functions, such as manipulating settings; navigating 
an options menu; performing mouse-like navigation on a screen 
of the separate electronic device; entering, manipulating, or 
deleting data in a personal information management applications- 
composing an email, telefax, or short message service (SMS) 
message; dialing -a telephone number; and the like. 

Preferably, a plurality of positions are detected by 
periodically detecting (e.g., numerous times per second) a 
portion of the address pattern that is currently adjacent to 
the optical sensor of the electronic pen. Accordingly, the 
detected positions provide an electronic representation of 
handwritten text or drawn images. The handwritten or drawn 
text or images can then be stored or converted into ASCII text. 
Furthermore, the information written or otherwise input using 
the electronic pen can be utilized by a user of a mobile phone, 
PDA, television, personal computer, or other electronic device 
to execute a function within the mobile phone, PDA, television, 
25 personal computer, or other device. 

In another aspect of the invention, when a portion of the 
address pattern is detected, the detected portion is compared 
with different areas of address patterns stored in a database 
memory to identify a function that corresponds to the detected 
portion of the address- pattern. Once a specific function is 
identified, a controller or processor performs the specific 
function . 
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In one embodiment, the electronic reading device stores 
and performs a few b<\sic functions corresponding to particular 
address patterns. In addition, instead of identifying and 
performing the function based solely on the detected pattern, 
5 the function can be initiated by writing a specific symbol 

either in any location or over a particular pattern. For 
example, the function might include storing or deleting 
information in the electronic reading device, sending 
information to another electronic device via a radio interface, 
10 and/or sending information* to an Internet address. 

In another embodiment, different areas of a specially 
formatted surface that includes various portions of an address 
pattern can be associated with different functions or 
applications. For example, the surface can be a paper having 
15 fields associated with: handwriting to text translation, 

language translation of messages, spell help, word translation 
wherein an audio translation is delivered to a mobile phone, 
accessing web pages, creating an electronic credit card 
receipt, sending information relating to a product or purchase, 
20 initiating a search on a search engine, and/or signature 

recognition. The particular function performed can be 
downloaded to the electronic reading device or to a related 
device for execution or might be executed in a remote server. 
Furthermore, the formatted surface might include 
25 preprinted icons associated with specific functions or with a 

retrieval of specific information. Moreover, sheets of paper 
that include different address patterns can be organized into 
a notebook, wherein a detection of a particular address pattern 
allows the particular sheet of paper to be identified. 

In yet another aspect of the invention, the present 
invention comprises a method and system for selecting a 
hyperlink and for providing a response to the selection. In 
particular, the hyperlink is printed, displayed, or otherwise 
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included on a formatted surface. The area of the formatted 
surface on which the hyperlink is printed or displayed includes 
a unique address pattern that is formatted such that a position 
on the address pattern can be identified by detecting a part of 
the address pattern with an electronic reading device. Once 
the electronic reading device detects the address pattern, a 
corresponding Internet address is identified, and a message 
indicating the detection/selection is sent to a server at the 
identified Internet address. The server then performs an 
operation that corresponds 'to the selected hyperlink, such as 
sending information to a mobile station, computer, or email 
address of the electronic reading device user in response to 
the message. 

Such -a method and system can be used in connection with 
hyperlinks that perform a wide variety of different functions. 
For example, such a hyperlink system can be used to obtain more 
information about a product through a link on the web; connect 
to a web page using a link contained in a book or newspaper; 
retrieve an electronic version of an identification or business 
card; retrieve accurate transportation timetable information; 
replace bar codes or magnetic strips; retrieve additional 
information about exhibits at a museum, zoo, or other 
attraction; automatically activate special office messages; 
automatically verify location for delivery persons; or identify 
and authenticate tickets. 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention, reference is made to the following detailed 
description taken in conjunction with the accompanying drawinas 
wherein : 

FIGURE 1 is a block diagram of a system in which an 
electronic pen can be used as an input device; 
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FIGURE 2 is a schematic diagram of a system for supporting 
use of the electronic pen described in connection with FIGURE 
1; 

FIGURE 3 is an illustration of the protocol stacks that 
can be used in the case of local communications between an 
electronic pen and an electronic pen client; 

FIGURE 4 is an illustration of protocol stacks that can 
be used when an electronic pen and an electronic pen client 
communicate with one another via an Internet connection; 

FIGURE 5 is an illustration of a protocol stack for 
communications between an electronic pen client and each of the 
supporting entities when the electronic pen client is not 
located within a server on the Internet; 

FIGURE 6 is an illustration of protocol stacks that are 
used for communications between an electronic pen client and 
each of the supporting entities when the electronic pen client 
is located on the Internet; 

FIGURE 7 is a block diagram of the electronic pen logic 
that handles positions, strokes, actions, and grid 
descriptions ; 

FIGURE 8 is a block diagram of a state machine for the 
electronic pen control block shown in FIGURE 7; 

FIGURE 9 is a block diagram of a state machine for an 
electronic pen client; 

FIGURES 10A-10C are a message flow and sxgnaling diagram 
illustrating the operation of the electronic pen system shown 
and discussed in connection with FIGURE 2; 

FIGURE 11 is an illustration of a reusable calendar 
reentry sheet; 

FIGURE 12 is an illustration of an example of a reusable 
phone settings entry sheet; 

FIGURES 13A and 13B are illustrations of an electronic pen 
10 having a paper touch sensor. 
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FIGURE 14 is a block diagram of some of the basic elements 
of the electronic pen in accordance with a preferred embodiment 
of the present invention; 

FIGURE 15 is an example of several areas having associated 
functions in accordance with one embodiment of the present 
invention; 

FIGURE 16 is an illustrative example of a credit card 
receipt for use in connection with the present invention; 

FIGURE 17 is an example of a preprinted paper for 
triggering selected functions in accordance with another 
embodiment of the present invention; 

FIGURE 18 is an illustrative example of an email form 
according to another embodiment of the present invention; 

FIGURE 19 is yet another illustrative example of a 
possible email form according to an alternative embodiment of 
the present invention; and 

FIGURE 20 is a schematic diagram of a hyperlink system in 
accordance with a preferred embodiment of the present 
invention . 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention relates to a system in which an 
electronic reading device, such as an electronic pen, an 
electronic mouse, or a hand scanner, works in cooperation with 
an address pattern (e.g., a specially formatted paper) to 
provide for a detection of a location of the electronic reading 
device over the address pattern. For instance, a pattern of 
dots can be defined such that, by examining a very small 
portion of the pattern, a precise location in the overall 
pattern can be determined. In fact, it is possible to define a 
pattern that has the size of 7 3,000,000,000, 000 A4 pages, which 
is equivalent to half the size of the entire United States. 
Portions of the pattern can be placed on sheets of paper or 
other objects. 
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Then, using an electronic scanner pen that can detect the 
dots in the pattern, it is possible to detect the location of 
the pen with respect to the unique pattern. For example, when 
such a pen is used Ln connection with a specially formatted 
paper, the pen can detect its position (e.g. , using a built in 
camera) by detecting a 3 mm by 3 mm portion of the pattern. By 
taking approximately 100 pictures per second, the pen is 
capable of determining its exact position to within 0.1 mm or 
less-. This system can be used to provide user input, to 
facilitate user interaction, or to store handwritten notes or 
drawings. Moreover, by associating portions of the overall 
pattern with certain applications, such a system can be used to 
interact with wide variety of applications. 

Referring now to FIGURE 1, there is illustrated an example 
of a system 2 in which an electronic pen can be used as an 
input device. The electronic pen includes an ink cartridge and 
is capable of writing in a typical fashion. The electronic pen 
10, however, includes some type of sensor (e.g., a built-in 
camera or optical detector) that is used for detecting an 
address pattern on a specially formatted piece of paper 12. In 
particular, the paper 12 is formatted with a small portion of 
a large address pattern (i.e., a portion of the address pattern 
is printed on the paper 12) such that when the electronic pen 
10 is used to write on or otherwise make marks on the paper 12, 
the writings or markings can be electronically detected and 
stored. 

As an example, the paper 12 might constitute a form that 
can be used for sending an e-mail. Thus, the paper 12 might 
include a space for writing in the e-mail address of an 
intended recipient, a space for writing a subject of the e- 
mail, and a space for writing the body of the e-mail. As the 
electronic pen 10 is used to fill in each of the spaces, the 
position and movement of the electronic pen 10 on the paper 12 
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can be determined by repeatedly detecting the current x, y 
coordinates of the pen 10 {e.g., at rate of 100 frames per 
second) . The markings can then be converted into ASCII text 
using an appropriate handwriting recognition program. Once the 
user completes the form, the e-mail can be sent, for example, 
by checking a send box at a predetermined location on the paper 
12. 

Preferably, the coordinate information collected by the 
pen 10 is sent by a short range radio transmitter in the 
electronic pen 10 to a nearby mobile station 14 using a short 
range radio interface 16 such as a local wireless radio link 
(e.g., a local wireless radio link supported by Ericsson's 
Bluetooth™ wireless communications technology) . Alternatively, 
instead of using a mobile station 14, rhe coordinate 
information could also be sent to, for instance, a desktop or 
portable computer, a personal digital assistant (PDA), a 
television, or a Bluetooth terminal. Moreover, instead of 
using a local wireless radio link, other types of local 
wireless links, such as Global System for Mobile Communication 
(GSM), such as inductive coupling and infrared light; other 
types of radio links; or wired transmission media, such as a 
cable can also be used. The information can then be forwarded 
via an appropriate link, such as a cellular air interface 13, 
to a base station 20 or other network node. 

As mentioned above, certain functions or applications can 
be associated with specific addresses or areas on the overall 
address pattern. The electronic pen 10 or an associated 
device, such as a mobile station 14, can then perform the 
function associated with an address when the address is read by 
the electronic pen 10. Assuming there are a large number of 
functions associated with different areas of the overall 
address pattern, there are a number of drawbacks to maintaining 
knowledge of all the various functions and their corresponding 
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addresses within the electronic pen 10 itself. For example, it 
would take a long time to define and develop in the electronic 
pen 10 the function or functions that correspond to a certain 
address space; memory demands and processing power requirements 
in the electronic pen 10 would be extensive; and special 
devices performing a limited number of specific applications 
(i.e, a subset of all of the applications) would most likely 
have to be developed. 

In accordance with the present invention, instead of 
having all functions and their corresponding addresses stored 
in the electronic pen 10, the electronic pen 10 has knowledge 
of only a few predetermined basic functions that allow it to 
perform a specific subset of functions. Other devices with 
which the electronic pen 10 can communicate (e.g., using a 
Bluetooth interface) can then perform the remaining functions. 
As an example, a particular address space can be associated 
with the function of sending information that has been written 
on an area of the address space to a URL on the Internet. The 
server associated with the URL can have the intelligence to map 
information written on certain areas of the address space to 
their associated functions or appliances. Alternatively, the 
URL server can store URL addresses to one or many other servers 
that map an address space to the function (s) or application { s ) 
associated with it. In another alternative, a particular 
address space can be associated with the function of sending 
information that has been written on an area of the address 
space to an electronic device, such as a mobile phone, laptap 
computer, or desktop computer, with which the electronic pen 10 
can communicate. By allowing one or more external devices to 
provide a mapping between a particular address space and the 
associated function or application, the system is very flexible 
and easily expandable. 
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At the same time, the electronic pen 10 only needs to host 
mappings of a few of the most common and/or basic functions or 
applications. Thus, the electronic pen 10 might perform a 
special, predetermined function (as defined within the 
electronic pen memory) when the electronic pen 10 reads a 
predefined small address space. Similarly, the electronic pen 
10 might perform a special, predetermined function (again as 
defined with the electronic pen memory) when a cross, circle, 
line, or other predetermined symbol is v/ritten on a predefined 
small address space. Either'of these actions might be used to 
perform such special functions a-s: (1) sending information (or 
a request for information) to a URL on the Internet; (2) 
sending information (or a request for information) to a mobile 
phone or other specified electronic device; (3) sending 
information to a URL in the mobile phone or other specified 
electronic device; (4) storing information in the electronic 
pen 10; or (5) deleting information from the electronic pen 10. 

Furthermore, the ability to perform certain functions in 
response to a written symbol is not necessarily limited to 
situations in which the symbol is written at a particular 
address space. Instead, the electronic pen 10 might be capable 
of recognizing special written patterns that trigger specific 
functions without requiring that the pattern be written in a 
preselected location on the address pattern. This pattern 
recognition capability can be supported either by instructions 
stored in the electronic pen memory or with the assistance of 
a server that is external (e.g., on the Internet) to the 
electronic pen 10 (e.g., using Internet web browsing 
protocols) . As an example: writing could launch a photo 

with an email; writing "^?" could be automatically converted to 
"Love from John;" writing could trigger retrieving an 

address book on a display, and writing " ///" could represent a 
"SEND" command. 



WO 01/61449 PCTYEP01/01398 

Referring now to FIGURE 14, there is illustrated a block 
diagram of some of the basic elements of the electronic pen 10 
in accordance with a preferred embodiment of the present 
invention. The electronic pen 10 includes a microprocessor 250 
that generally controls the operation of the electronic pen 10. 
In particular, the microprocessor 250 controls coirtmunica t ions 
with other electronic devices using a Bluetooth transceiver 252 
and receives coordinates detected by a pattern sensor 256. The 
microprocessor 250 further compares position data received from 
the pattern sensor 256 with predetermined symbols or patterns 
stored in a function memory 254. When a detected symbol or 
pattern matches a symbol or pattern, a function stored in the 
function memory 254 that corresponds to the detected symbol or 
pattern is- executed by the electronic pen 10. 

Some functions or applications are preferably handled by 
a device external to the electronic pen 10, such as a server on 
the Internet. In accordance with the present invention, when 
the electronic pen 10 writes on a specific portion of the 
address pattern, an application or web page, with or without 
scripts, is automatically executed or downloaded to the 
electronic pen 10 or to another device associated with the 
electronic pen 10. Referring now to FIGURE 15, there is 
illustrated an example of several areas having associated 
functions in accordance with one embodiment of the present 
invention. Handwritten messages or notes written using the 
electronic pen 10 can be converted to text using a Handwriting 
to Text Conversion Area 260. Once the handwritten message is 
complete, the user touches the electronic pen 10 to a 
designated "Convert" area 262, which causes the hand-written 
document to.be sent to a server that performs the conversion. 
The resulting text could then be stored in a database or 
delivered to an electronic device (e.g., a mobile phone or 
personal digital assistant) for storage and/or display. 
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Similarly, translations of handwritten words or phrases 
can be obtained using a predefined Translation Area 264. After 
the word or phrase is written into the area 264 , the user 
selects the desired language for translating to by touching the 
desired language in a language selection area 266 and requests 
the translation by touching the electronic pen 10 to a 
designated "Translate" command area 268. The result can then 
be sent, for example, to the user's mobile phone 14 and 
displayed as text or played as audio. Other analogous 
functions include obta ining* def ini t ions of words, retrieving 
thesaurus entries, or obtaining spelling help. 

Another possible function is obtaining access to a web 
page by writing a complete or abbreviated URL in a web access 
area 270 and touching a "Connect" command area 272. For 
instance, writing "YAHOO" in the web access area 270 will cause 
a mobile phone or laptop in communication with the electronic 
pen 10 to automatically access the YAHOO! web page. Finally, 
writing a term or terms in a search area 274 and touching a 
"Search" command field 276 initiates a search engine, which 
responds by providing search results to a phone or laptop that 
is in communication with the electronic pen 10. 

Referring now to FIGURE 16, there is depicted an 
illustrative example of a credit card receipt 280 for use in 
connection with the present invention. The credit card receipt 
280 includes a portion of the address pattern reserved for 
credit card receipts. When a total amount is entered in the 
"Total Amount" field 284 and the user signs the receipt 280 in 
the "Signature" field 282 using the electronic pen 10, a 
digital image of the signature and sum are automatically sent 
to the credit card company by an application stored within the 
electronic pen 10 or in a server that is accessed by the 
electronic pen 10. 
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Yet another example of a function that can be performed 
by reading a particular portion of the overall address pattern 
is retrieval of product information. When the electronic pen 
10 reads a particular pattern (e.g. , on a product package or in 
an advertisement) , a request can be sent to a remote server, 
which might respond to the request by sending product 
information to a mobile phone, PDA, television or computer 
associated with the user of the electronic pen 10. In 
addition, information might be sent to a retailer or 
manufacturer. 

Referring now to FIGURE 17, there is illustrated an 
example of a preprinted paper 290 for triggering selected 
functions in accordance with another embodiment of the present 
invention.. By writing in a written input field 300 and then 
touching the electronic pen 10 to an address field 298 (or by 
writing in a field dedicated to address inputs), an address 
recognition function might be triggered. Similarly, written 
input can be inserted in an email or a reply email by touching 
the pen 10 to an email field 294 or a reply field 296, 
respectively. The preprinted paper might also include an icon 
field 292 that contains one or more icons that, when touched or 
clicked by the electronic pen 10, trigger a connection with a 
particular web address. Such icons can be sold, for example, 
to advertisers by the producer of the preprinted paper 290 or 
the owner of a server that is associated with the portion of 
the address pattern included on the paper 290. 

In one embodiment of the invention, preprinted paper 290, 
or paper that is otherwise allocated to particular functions, 
can be organized in a notebook that can include pages having 
multiple different functions and/or multiple pages having the 
same function (e.g. , it would be desirable to have a number of 
pages dedicated to sending an email to allow for many emails to 
be sent) . The notebook and/or individual pages in the notebook 
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can be identified by the address pattern printed thereon. 
Alternatively, individual pages might be distinguished by 
manually inserting a page number or name. Once identified, the 
page triggers an associated function or application in, for 
example, the electronic pen 10, in a mobile phone 14, or in a 
remote server. Preferably, the electronic pen 10 or a server 
monitors paper usage and prompts the user to buy new paper 
(e.g., by sending a message to the user on a phone or laptop 
with information from the paper manufacturer) . In addition, an 
icon in the notebook might Automatically initiate web access 
to a server where additional paper can be ordered or purchased. 

Referring now to FIGURE 18, there is illustrated an email 
form 310 according to another embodiment of the present 
invention. Software in a server associated with the form 310 
recognizes the form as an email form based on the underlying 
address pattern. The software further recognizes various 
fields on the email form 310. The page is identified by a page 
identification field 312, which distinguishes the page from 
other email forms 310. Several special function fields 314 are 
also included. When the electronic pen 10 is used to touch or 
"click on" one of these fields 314, the corresponding function 
can be performed. By writing a message and identifying an 
addressee or addressees in message field 316 and address fields 
318, the server software will call a handwriting recognition 
application to convert the message into text and the address 
into a known email address. A similar form might also be used 
for other types of Intranet or Internet messaging with another 
individual or with a group (e.g., a chat area between a user 
and his or her secretary, a telefax, or a short message service 
(SMS) message) . 

Yet another example of a possible email form 320 is 
depicted in FIGURE 19. This form 320 also includes a message 
field 322, a variety of special function fields 324, and an 
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address field 326. ]n this embodiment, however, the email form 
320 includes a number of shortcut fields 328. By simply using 
the electronic pen 10 to touch or "click on" one of the 
shortcut fields 328, a word, image, number, symbol, or address 
can be inserted into the message field 322 or address field 
326. 

In one aspect of the present invention, new functions can 
be implemented simply by defining functions or applications 
that correspond to new portions of the overall address pattern. 
Such an implementation does 'not require a creation of any new 
protocols between the electronic pen 10 and an external server. 
One example of a new function includes using an address pattern 
to facilitate detection of whether a product is used or. has 
been tampered with. If a package is opened, then a certain 
area that includes an address pattern is mechanically or 
chemically destroyed. If the address pattern cannot be 
detected by the electronic pen 10, then it is known that the 
product has been used or tampered with (e.g., for use in 
connection with airtight packages) . Another example of a new 
function is for matching parts in an assembly. Thus, the parts 
of a house or table include addressed areas that can be 
detected using an electronic pen 10 to determine how and if the 
parts are designed to be attached together. An electronic pen 
10 and addressed area could also be used for alcohol testing by 
testing whether a person can draw a straight line or follow a 
pattern to determine whether he or she is drunk. 

In another example, an enlarged pattern could be painted 
on the ground so that a car, robot, or other moving object can 
determine its position. This could be used, for instance, for 
automatic robots that retrieve objects from shelves in a 
warehouse. Another possible new function is to use an 
electronic pen 10 and formatted surface for instantly scoring 
a school test, drivers' license test, etc. Answers selected or 
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written by a test taker could be sent directly to a PC that 
automatically scores the answer. Similarly, a form could be 
used for statistics or voting, wherein a user's selection is 
sent directly to a server on the Internet for computing 
statistics or tabulating voting results. Finally, a particular 
part of the address pattern could be used so that multiple 
persons can enter information into a single electronic 
document. For instance, 'family members could each have their 
own paper that allows them to enter items in a common shopping 
list that can be viewed on" a mobile station 14 or PC. The 
invention also allows scripts used for implementing functions 
and applications to be executed either in the electronic pen 10 
or in an external server. 

Referring now to FIGURE 2, there is illustrated a 
schematic diagram of a system 2 for supporting use of the 
electronic pen 10 described in connection with FIGURE 1. 
Throughout the subsequent discussion, the system 2 is described 
primarily in connection with an electronic pen 10. It will be 
understood, however, that the invention and the underlying 
system 2 can instead use any type of electronic reading device, 
such as an electronic pen, an electronic mouse, or a hand 
scanner. As shown in FIGURE 2, the system 2 includes six 
different entities, including the electronic pen 10, electronic 
pen client 22, a control node 24, a name server 26, a base 
translator 28, and an application server 30. Although these 
various devices are described and depicted separately, it is 
also possible to combine two or more of the entities into the 
same device (e.g., the electronic pen 10 and electronic pen 
client 22 can be contained in the same device) . 

The electronic pen 10 is responsible for detecting 
positions on the address pattern, producing actions, and 
sending information to the electronic pen client 22. In 
addition to being able to leave pen markings, some electronic 



17 



WO 01/61449 PCT/EP01/01398 

pens can also have the ability to produce' other types of 
output, such as .sound, vibration, or flashing lights. The 
electronic pen 10 includes a memory for storing a current grid, 
which comprises information relating to an area of the address 
pattern that is near the most recently detected position of the 
electronic pen 10. When the electronic pen 10 is loaded with 
the current grid, it knows what actions to take based on the 
positions that are read from the address pattern. When the 
electronic pen 10 is first turned on or when it moves to an 
area outside of the current* grid, the electronic pen 10 must 
first request a new grid description before it can continue 
processing information. In such a situation, the electronic 
pen 10 requests a new grid description from the electronic pen 
client 22... 

The electronic pen client 22 can be located in a mobile 
station 14, in a PDA, in a television, in a desktop or portable 
computer, in the electronic pen 10 itself, in a server 
somewhere on the Internet, or in another device. The 
electronic pen client 22 serves as the center of communications 
in the overall system 2. In particular, the electronic pen 
client 22 receives new grid requests and action requests from 
the electronic pen 10 and responds to these requests by- 
contacting an appropriate entity withxn the overall system 2 to 
properly respond to the request from the electronic pen 10. 
Furthermore, when the electronic pen 10 is being used in 
connection with a particular application, the electronic pen 
client 22 can store the application and/or any corresponding 
data received from the electronic pen 10 to facilitate 
processing and use of the application. 

The name server 26 is used for translating a detected 
position on the address pattern into a Uniform Resource 
Location (URL) associated with that position. Different 
portions of the address pattern are assigned to different 
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applications. Neither the electronic pen 10 nor the electronic 
pen client 22 , however, is aware of all of the different 
applications and the particular areas assigned to each 
application. Thus, when the electronic pen 10 detects a new or 
unknown position, it forwards the position information to the 
electronic pen client 22, which in turn sends the information 
to the name server 26. The name server 26 then identifies an 
application associated with the received position and retrieves 
a URL where a description of the particular application can be 
found. The retrieved URL can then be used by the electronic 
pen client 22 to retrieve the application description. 

As an alternative, the name server 26 can comprise a 
global name server that keeps track of a location, in the form 
of URLs to .local name servers, where more information can be 
found about different addresses in the pattern. Similarly, 
each local name server can use other local name servers to 
obtain the necessary information, i.e. , to convert a position 
into a URL where an application description can be found. At 
the lowest level, the local electronic pen client should know 
all the paper addresses that are within a specific application 
or applications. 

There are some services that should be available in the 
overall system 2 for which it is inconvenient or not feasible 
to support such services in the electronic pen 10 ox the 
electronic pen client 22. In such a case, the base translator 
28 can be used to support the services. For example, the base 
translator 28 might contain handwriting recognition software 
for converting pen actions into text or for converting pen 
actions into a predefined set of symbols. When such services 
are needed, the electronic pen client 22 can send a request to 
the base translator 28 along with the necessary data, and the 
base translator 28 can perform the requested service. 
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Another entity in the system 2 is a control node 24. The 
control node 24 i > used for responding to actions in a 
standardized way. For example, the control node 24 can fee used 
to respond to certair generic functions, such as "cancel" or 
"submit" functions, xn a consistent manner without regard to 
the particular application that is currently active. 

In addition, the control node 24 is used for creating 
streaming-like applications. For instance, some applications 
might require that the positions on the address pattern that 
are detected by the electronic pen 10 be immediately sent, upon 
detection, to the electronic pen client 22 for use by the 
application (i.e., the electronic pen 10 does not wait to 
transmit the position data until a complete stroke is detected 
or until a "send 7 ' field is touched) . One example is an 
application that is used to control an industrial robot in a 
warehouse. In such a case, the application description that is 
loaded onto the electronic pen server 22 can include 
instructions that all positions be streamed to a control node 
24. As a result, the control node 24 can receive the positions 
in real time and can control the robot without waiting for the 
form (i.e., the current grid) to be completed. Thus, the 
control node 24 can perform a real-time translation from 
detected positions to a responsive action, such as moving an 
object (e.g., a robot, a valve, etc.) or controlling a process. 

The application server 30 is a regular web or wireless 
application protocol (WAP) server that supports an application 
associated with a particular area of the address pattern. The 
application server 30 stores an application description and 
provides the application description to the electronic pen 
client 22 upon request. In addition, the application server 30 
receives input data from the electronic pen 10 via the 
electronic pen client 22. For example, the application 
description might define a number of data entry areas on a 
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form. Thus when data is entered on the form by the electronic 
pen 10, the data is received by the electronic pen client 22, 
converted into text using handwriting recognition software, and 
forwarded to the application server 30, which stores the data 
or otherwise processes the data in accordance with the function 
of the application 

Referring now to FIGURES 3 through 6 there are illustrated 
various examples of protocol stacks that can be used for 
communicating between the entities shown in FIGURE 2. 
Generally, however, such protocols apply however, only if the 
two communicating entities are implemented in different 
devices. If two or more entities are combined into one device, 
a proprietary protocol can be used to communicate between the 
entities. ..FIGURE 3 illustrates the protocol stacks that can be 
used in the case of local communications (e.g., using 
Bluetooth™) between the electronic pen 10 and the electronic 
pen client 22. If, on the other hand, the electronic pen 10 
and the electronic pen client 22 communicate with one another 
via an Internet connection, the protocol stacks depicted in 
FIGURE 4 will be used. FIGURE 5 illustrates a protocol stack 
for communicating between the electronic pen client and each of 
the supporting entities, such as the name server 26, the 
control node 24, the base translator 28, and the application 
server 30, when the electronic pen client 22 is not contained 
within a server on the Internet (e.g., such as when the 
electronic pen client 22 is located in a mobile phone 14). 
Finally, FIGURE 6 depicts the protocol stacks that are used 
when the electronic pen client 22 is located on the Internet 

There are a number of procedures that can be used by the 
various entities in the syscem 2 to allow the system Co operate 
properly. When the electronic pen 10 detects a position on the 
address pattern that is not within its currently loaded grid or 
when the electronic pen 10 has no currently loaded grid, the 
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electronic pen 10 initiates a new grid procedure. The new grid 
procedure involves sending a new grid request object to the 
electronic pen client 22. The new grid request object contains 
the newly detected position, a description of the actions that 
the electronic pen 10 can natively support, and a description 
of the output signals that the electronic pen 10 supports. The 
reply to a new grid request object is a grid description, which 
can be provided by the electronic pen client 22 from its own 
internal memory or from the information provided by an 
application server 30. Generally, the electronic pen client 22 
extracts the grid description from an application description 
received from the application server 30. The grid description 
should only contain action-field-types that the electronic pen 
10 has indicated that it natively supports, which means that 
the electronic pen client 22 in some cases should convert the 
extracted grid description into a format that the electronic 
pen 10 can understand. 

In some situations, it may be necessary for the electronic 
pen 10 to unload its current grid at the request of the 
electronic pen client 22. In such a case, the electronic pen 
client 22 sends an empty grid description to the electronic pen 
10, thereby causing the electronic pen 10 to unload its current 
grid. This can occur, for example, when a particular 
application is complete or when a new grid description request 
received from the electronic pen 10 cannot be fulfilled, such 
as when the position received from the electronic pen 10 is not 
registered in the name server 26. 

Another similar message is the empty grid description with 
a grid exception. When the electronic pen 10 requests a new 
grid description from the electronic pen client 22, the 
electronic pen client 22 uses the detected position specified 
in the request to ask the name server 26 for a URL where the 
application description can be found. If no URL is returned, 
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the electronic pen client 22 can send an empty grid description 
with a grid exception to the electronic pen 10. The grid 
exception comprises a rectangle or other shape indicating the 
area around the detected position where no registered 
applications can be found. Preferably, the indicated area is 
as large as possible so that the electronic pen 10 and/or 
electronic pen client 22 know the extent of the surrounding 
area that is unassigned and do not have to repeatedly send 
requests to the name server 26. Thus, the empty grid 
. description with a grid exception causes the electronic pen 10 
to unload its current grid and also informs the electronic pen 
10 of an area surrounding the detected position that can 
essentially be ignored because its is not associated with any 
application,. 

The procedure that is used when the electronic pen 10 
detects a new position is a find application description 
location procedure. This procedure is used by the electronic 
pen client 22 to translate a detected position received from 
the electronic pen 10 into a URL where a description of an 
application corresponding to that position can be found. The 
procedure involves sending a request from the electronic pen 
clrent 22 to the name server 26 containing identification of 
the detected position. The name server 26 responds by sending 
a reply to the electronic pen client 22 containing a URL where 
an application description can be found or, if the detected 
position is not registered in the name server 26, containing an 
. indication that no associated application is known to exist. 
Once the electronic pen client 22 knows the URL where an 
application description can be found, the electronic pen client 
30 22 can initiate a get application description procedure, which 

allows the electronic pen client 22 to retrieve the application 
description from the application server 30. In particular, the 
electronic pen client 22 sends an application description 
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request containing e unique ID for the requesting electronic 
pen 10 and/or electronic pen client 22 to the application 
server 30 located at the URL address provided by the name 
server 26. In response, the application server 30 provides an 
application description object to the electronic pen client 22, 
which loads the application onto the electronic pen client 22. 
The application description object is similar to an HTML form 
with some additions and modifications. 

Furthermore, the application description object can be 
sent from the application 'server 30 to the electronic pen 
client 22 in response to a submitted form (i.e., a submission 
of one completed form might automatically result in a new form 
being loaded onto the electronic pen client 22) . A related 
procedure is the application submit procedure, which is used by 
the electronic pen client 22 when the user of the electronic 
pen 10 selects a "submit" field in a form. In response to the 
selection of the * submit" field, the electronic pen client 22 
will submit the form content in accordance with instructions 
received in the application description. Typically, the 
electronic pen client 22 will submit the form content, in the 
same way as a regular web browser, to a URL specified in a form 
tag of the application description. 

When an action that can be handled by the electronic pen 
10 itself is generated, an action procedure is initiated by the 
electronic pen 10 to send an action request object to the 
electronic pen client 22. If the electronic pen client 22 
cannot translate the action into a field value itself, the 
electronic pen client 22 further forwards the request to a base 
translator 28 for translating the action into a field value. 
In response to the action request object, .an action reply 
object is sent from the electronic pen client 22 to the 
electronic pen 10. The action reply object contains output 
information that indicates to the electronic pen 10 which 
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outputs signals to use. The output information, however, 
cannot be of type that the electronic pen 10 has previously 
indicated that it does not support. In some instances, the 
action reply object might contain a new grid description. In 
such a case the electronic pen 10 will unload its current grid 
description and load the new grid description. Similarly, xf 
the action reply object contains an empty grid description, the 
electronic pen 10 will simply unload its current grid 
description. 

The action request object is also sometimes used to 
specify actions that should be processed by the control node 
24. In this instance, the electronic pen client 22 initiates 
a control procedure by forwarding the received action to the 
appropriate control node 24. As a result, the control node 24 
15 sends an action reply object to the electronic pen client 22. 

The operation of the electronic pen 10 will now be 
discussed in greater detail. Each electronic pen 10 has a 
unique pen ID, which is sent to the application server 30 when 
an application description is requested. The electronic pen ID 
allows the application to identify the particular user that is 
using the application and to distinguish between multiple 
concurrent users of the same application, such as when 
different electronic pens 10 are being used in connection with 
separate sheets of paper that each contain the same portion of 
25 the address pattern. 

Referring now to FIGURE 7, there is illustrated a block 
diagram of the electronic pen processor logic that handles 
positions, strokes, actions, and grid descriptions for the 
electronic pen 10. The electronic pen 10 includes a control 
block 32 for, controlling the operation of the electronic pen 
10. A grid description block 34 represents a memory location 
that stores a current grid description. At any given time, the 
electronic pen 10 can be in either of two modes. In a first 
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mode, a grid description is loaded, while in a second mode, the 
grid description block 34 is not loaded with a current grid 
description . 

As the electronic pen 10 moves across an address pattern, 
the electronic pen 10 periodically (e.g., every 1/100 of a 
■ second) detects a position by detecting all of the dots within, 
for example, a 3mm by 3mm area. Each detected position is 
forwarded (as indicated at 36) to a position first in first out 
(FIFO) block 38, which acts as a buffer for temporarily storing 
the detected positions. The clocking of the position FIFO 
block 38 is controlled by the control block 32 (as indicated at 
40) . 

The detected position is fed from the position FIFO block 
38 (as indicated at 42) to an in grid detector 44. The in grid 
15 detector 44 retrieves data from the grid description block 34 

(as indicated at 46) and determines whether the received 
position is within the loaded grid description. If not, the in 
grid detector 44 notifies the control block 32, which in turn 
initiates a request for a new grid. When the detected position 
is within the current grid, the position is then sent (as 
indicated at 50) from the in grid detector 44 to a stroke 
engine 52. The stroke engine 52 converts the received 
positions into strokes, which are then sent (as indicated at 
54) to an action engine 56. A complete stroke is created when 
25 the electronic pen 10 is lifted from the paper or when it moves 

outside of the grid field where the stroke began. Finally, the 
action engine 56 converts the received stroke into an action 
that can be sent to the electronic pen client 22. By using 
grid action-field-types, the action engine knows which type of 
action to produce for a specific grid field. 

Referring now to FIGURE 8, there is illustrated a block 
diagram of a state machine for the control block 32 shown in 
FIGURE 7. In this figure, events are indicated in capital 
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letters, while tasks associated with the event are depicted in 
brackets. The process starts at step 60 with a start up event 
62, which causes the position FIFO block 38 to begin receiving 
detected positions. Initially, the electronic pen 10 is in a 
no grid loaded state 64, which means that the electronic pen 10 
does not have a grid loaded in the grid description block 34. 
As a result, the control block 32 generates an outside grid 
indication 66, thereby causing the electronic pen 10 to send 
the request for a new grid description to the electronic pen 
client 22 (i.e. , in accordance with the new grid procedure) and 
to stop the FIFO buffer 38. At this point, the electronic pen 
10 enters a waiting for grid state 68. 

Once the new grid has been received (as indicated at 70) , 
the control block 32 moves to a grid loaded state 72, at which 
time the new grid is loaded into the grid description block 34 
and the position FIFO block 38 resumes operation. On the other 
hand, if no grid is received (as indicated at 74), at least a 
portion of the positions stored in the FIFO buffer 38 are 
erased. Which part of the FIFO buffer to erase is determined 
by the grid exception area, if any, in the received empty grid 
description. Accordingly, all positions stored in the FIFO 
buffer 38 that are within the grid exception area should be 
erased. If no grid exception is received, the stroke 
associated with the position is erased. In addition, the FIFO 
block 38 resumes operation and the control block 32 moves into 
the no grid loaded state 64. 

When the control block 32 is in the grid loaded state 72, 
a current grid is loaded in the grid description block 34. 
While the control block 32 remains in this state 72, the 
position FIFO block 38 continues to receive detected positions 
and passes them on to the stroke engine 52 and action engine 
56. Actions produced by the action engine 56 are sent (as 
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indicated at 58) to the electronic pen client 22 (i.e., in 
accordance with the action procedure described above) . 

At some point, an outside grid indication 74 may be 
received by the control block 32 from the in grid detector 44. 
The outside grid event 74 causes the FIFO block 38 to stop 
generating new positions. In addition, the electronic pen 10 
enters a flushing stroke and action state 76 wherein the 
strokes that are currently in the stroke engine 52 and the 
actions that are currently in the action engine 56 are flushed 
to the electronic pen client'22. Once the stroke engine 52 and 
action engine 56 have been fully flushed (as indicated at 78) , 
the electronic pen 10 sends a request for a new grid to the 
electronic pen client 22 and unloads the current grid. The 
control block 32 then moves back into the waiting for grid 
state 68 . 

As a general matter, the electronic pen 10 may be capable 
of supporting various different types of output, including 
audio, such as warning tones; visual, such as a flashing light; 
tactile, such as vibration; and/cr ink. In some cases, it might 
be desirable to allow the user of the electronic pen 10 to turn 
off the ink of the pen 10, such as when the electronic pen is 
being used on a portion of the address pattern that is public 
or shared or when the user wants to be able to reuse the 
current sheet of paper. 

The electronic pen client 22 will now be described in 
greater detail. Generally, the electronic pen client 22 is 
analogous to a regular web browser. It is responsible for 
loading applications from application servers 30 and for 
handling input form the electronic pen 10. Preferably, the 
electronic pen client 22 is located in a separate device-f rom 
the electronic pen 10 itself. This is because it is desirable 
to minimize the size and power supply requirements of the 
electronic pen 10, which will likely be adversely affected by 
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the processing resources and memory necessary to support the 
functions of the electronic pen client 22. 

Referring now to FIGURE 9, there is illustrated a block 
diagram of a state machine for the electronic pen client 22. 
Initially, the electronic pen client 22 is in a no application 
loaded state 80. The electronic pen client 22 recognizes only 
one signal when in this state 80, namely a new grid request 
from the electronic pen 10. Such a request causes a load grid 
indication event 82. The electronic pen client 22 responds by 
sending a request to the name'server 26 to translate a position 
contained within the new grid request into a URL where the 
application description can be found (i.e., in accordance with 
the find application location procedure) . Next, the electronic 
pen client: 22 enters a waiting for application description URL 
state 84. If no URL for the application description can be 
found (as indicated at 86) , the electronic pen client 22 sends 
a new grid reply to the electronic pen 10, wherein the reply 
contains an empty grid description with a grid exception. As 
a result, the electronic pen client 22 returns to the no 
application loaded state 80. 

If a URL for the application description is received from 
the name server 26 (as indicated at 88) f the electronic pen 
client 22 sends a request to the application server 30 to 
retrieve the application description (i.e. , in accordance with 
the get application description procedure) . Accordingly, the 
electronic pen client 22 enters a waiting for application 
description state 90. 

If the electronic pen client 22 does not receive an 
application description from the application server 30 (as 
indicated at 92), a new, grid reply is sent by the electronic 
pen client 22 to the electronic pen 10 wherein the reply 
contains an empty grid. Thus, the electronic pen client 22 
returns to the no application loaded state 80. If, however, 
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the electronic pen client 22 does receive an application 
description from the application server 30 (as indicated at 
94) , the electronic pen client 22 sends a new grid reply to the 
electronic pen 10 containing a new grid description, and the 
electronic pen client 22 loads the application in its memory. 
In addition, the electronic pen client 22 moves into an 
application loaded state 96. 

In the application loaded state 96, five types of actions 
can be received by the electronic pen client 22 from the 
electronic pen 10. First, *a received action can include a 
request that the electronic pen client 22 cannot handle itself, 
in which case the electronic pen client 22 will send the action 
to the base translator 28 (as indicated at 98) . The electronic 
pen client 22 then moves into a waiting for response from the 
base translator state 100. Once a base translator response 102 
is received by the electronic pen client 22, the electronic pen 
client 22 updates a current form or other data associated with 
the currently loaded application and sends an action reply to 
the electronic pen 10 with appropriate output information. 

Another type of action that the electronic pen client 22 
can receive from the electronic pen 10 is a request that should 
be forwarded to a control node 24. In such a case, the action 
is sent to a control URL specified in the application 
description (as indicated at 104), and the electronic pen 
client 22 enters a waiting for response from the control state 
106. Once a response is received from the control (as 
indicated at 108), the electronic pen client 22 sends an action 
reply to the electronic pen 10 with appropriate output 
information. 

A third type of action is a submit form request, in 
response to which the electronic pen client 22 will submit the 
current form to the application server 30 that is identified by 
the URL in the application description (as indicated at 110) . 
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The electronic pen client 22 then enters a waiting for response 
from the application server state 112. If the application 
server 30 responds by sending an empty application description 
to the electronic pen client 22 (as indicated at 114), the 
current application is unloaded from the electronic pen client 
22 and an action reply is sent to the electronic pen 10 with an 
empty grid. As a result, the electronic pen client 22 returns 
to the no application loaded state 80- On the other hand, if 
the application server 30 responds with a non-empty application 
description, the old application is unloaded from the 
electronic pen client 22, the new application description is 
parsed and loaded in the electronic pen client 22, an action 
reply is sent to the electronic pen 10 with a new grid 
description and with appropriate output information, and 
finally the electronic pen client 22 returns to the application 
loaded state 96. 

A fourth type of action that can be received by the 
electronic pen client 22 from the electronic pen 10 is a 
request to load a new grid. This action occurs, for example, 
when a position outside of the current grid is detected by the 
electronic pen 10. When a new grid request is received, the 
electronic pen client 22 sends a request to the name server 26 
(as indicated at 116) and the electronic pen client 22 returns 
to the waiting for application description URL state 84. 

Finally, a fifth type of action that can be received by 
the electronic pen client 22 is an action that the electronic 
pen client 22 can handle itself, in which case the electronic 
pen client 22 updates the current form and sends an action 
reply to the electronic pen 10 with appropriate output 
information (as indicated at 118) . The electronic pen client 
22 then remains in the application loaded state 96. One type 
of action that the electronic pen client 22 might be able to 
handle itself is a local application. For example, the 
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electronic pen clien : 22 might be capable of performing certain 
basic functions that are defined by a local application. Thus, 
when the electronic pen client 22 receives a new grid request, 
the position ■ associated with the new grid request can be 
analyzed to determine if it corresponds to a local application, 
if so, the electronic pen client 22 can load the application 
description from its local memory, send a new grid description 
to the electronic pen 10 without having to communicate with the 
name server 26 or the application server 30. 

Another action that might be handled locally by the 
electronic pen client 22 relates to the selection of fields 
within a form. When the electronic pen client 22 receives an 
action, the field that corresponds to that action receives 
focus. When this occurs, the electronic pen client 22 might 
display the field's value on its display or output the value by 
audio. In addition, the electronic pen client 22 might allow 
the user to edit the value of the field by means other than the 
electronic pen 10. Yet another type of action that might be 
handled by the electronic pen client 22 itself are actions that 
relate to a clipboard function. When a ^copy" field is 
selected, the value of the field that had focus at the time the 
copy field was selected is transferred to the clipboard. 
Similarly, when a '"paste" field is selected, the value stored 
in the clipboard is transferred to the field that had focus at 
the time the paste field was selected. 

Referring now to FIGURES 10A through 10C, there is shown, 
by way of example, a message flow and signaling diagram 
illustrating the operation of the electronic pen system 2 
depicted in and discussed in connection with FIGURE 2. 
Initially, the electronic pen 10 detects a first position on 
the address pattern at step 120 (e.g., at a location on a sheet 
of paper designated for composing and sending e-mails) . At 
this stage, it is assumed that the electronic pen 10 is in a no 
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grid loaded state. Thus, in response to the detection of the 
first position, the electronic pen 10 sends a new grid request 
122, which contains the detected position information, to the 
electronic pen client 22. As a result, the electronic pen 
client 22 sends an application location request 124 containing 
the detected position information to the name server 26, at 
step 126. The name server 26 translates the detected position 
into a URL where an application description that corresponds to 
the detected position can be found (e.g., a URL address for a 
server containing an e-marl application) , and returns an 
application location reply 128 containing the retrieved URL to 
the electronic pen client 22. 

The electronic pen client 22 then sends an application 
description request 130, which contains the unique pen ID for 
the electronic pen 10, to the application server 30. The 
application server 30 retrieves the application description at 
step 132 and sends an application description reply 134 
containing the retrieved application description to the 
electronic pen client 22. The electronic pen client 22 then 
parses and srores the application description at step 136. 
This step further involves generating a current grid 
description from the application description and sending the 
grid description to the electronic pen 10 in a new grid reply 
138. The electronic pen 10 stores the received grid 
description at step 140 and resumes processing of the detected 
positions. Using the detected positions and the information in 
the grid description (e.g., so that the electronic pen 10 knows 
which fields of the e-mail form are being filled in) , the 
electronic pen 10 generates strokes at step 142 and generates 
actions at step 144 using the stroke engine 52 and action 
engine 56 shown in FIGURE 7. 

Each time an action is generated that cannot be handled 
by the electronic pen 10 itself, an action request 146 
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containing a description of the action is sent from the 
electronic pen 10 to the electronic pen client 22. At this 
point, the electronic pen client 22 should determine what type 
of action has been received so that it can respond to the 
action in an appropriate manner. First, it is determined 
whether the action requires the attention of, or otherwise 
should be processed in accordance with, a local application at 
step 148. Very basic applications or frequently used 
applications (e.g. , delete entered text) , for example, might be 
stored locally to avoid having to contact another entity. In 
such a case, the electronic pen client 22 retrieves the local 
application at step 150 and sends an action reply 152, which 
can contain a new grid description or other appropriate 
information . 

However, if it is determined at step 148 that the received 
action does not relate to a local application, the process 
continues at step 154 where it is determined whether the 
received action requires processing by an external translator 
(e.g., handwriting recognition) . If so, an action request 156 
containing a description of the action is sent by the 
electronic pen client 22 to the base translator 28. The base 
translator 28 processes the action at step 158 and sends an 
action reply 160 containing output information responsive to 
the received action (e.g., text corresponding to written 
characters) to the electronic pen client 22, which can forward 
the output information to the electronic pen 10 in an action 
reply 162, if necessary. 

If it is determined at step 154 that the received action 
does not require processing by an external translator, it is 
next determined whether the action relates to a control 
application at step 164. If so, an action request 166 
containing a description of the action is sent by the 
electronic pen client 22 to the control server 24. The control 
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server 24 processes the received action at step 168 and, if a 
response is necessary, returns output information responsive to 
the received action in an action reply 170, which is forwarded 
from the electronic pen client 22 to the electronic pen 10 in 
5 an action reply 172. 

Assuming that it is determined at step 164 that the 
received action does not relate to a control function, it is 
next determined whether the action comprises a request to 
submit a form at step 174 (e.g., a selection of a "send" area 
10 on the e-mail form) . If so, an action request 176 containing 

the data entered onto the form is sent by the electronic pen 
client 22 to the application server 30. The application server 
30 processes the form at step 178 and sends an action reply 180 
containing - a new application description (or an empty 
15 application description) to the electronic pen client 22. The 

electronic pen client 22 parses and stores the new application 
description at step 182 and generates a new grid description 
from the newly received application description. The 
electronic pen client 22 then sends an action reply 184 
20 containing the new grid description. Although not illustrated 

in the figure, the electronic pen 10 will typically respond to 
the receipt of a new grid description by unloading its current 
grid description and loading the new grid description into its 
memory . 

25 At some point, it is assumed that the electronic pen 10 

detects a position that is outside of the currently loaded grid 
at step 186. In response to such an event, the electronic pen 
10 sends a new grid request 138 containing the newly detected 
position data to the electronic pen client 22. In response, 

30 the electronic pen client 22 again generates an application 

location request 190 containing the detected position data and 
sends the request to the name server 26. The name server 26 
determines whether a URL for an application description that 
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corresponds to the nevly detected position is available at step 
192 . 

If so, the name server 26 sends an application location 
reply 194 containing retrieved URL to the electronic pen 
client 22, which in turn sends an application description 
request 196 containing the unique pen ID for the electronic pen 
10 to the application server 30 at the identified URL address, 
just as previously discussed in connection with messages 128 
and 130. In this case, however, it is assumed that the 
application server 30 determines that the requested application 
description is unavailable at step 198. As a result, the 
application server 30 sends an application description reply to 
the electronic pen client 22 containing an empty application 
description. In response to the receipt of an empty 
application description, the electronic pen client 22 unloads 
the current application at step 202 and sends a new grid reply 
204 containing an empty grid description to the electronic pen 
10. The electronic pen 10 responds to the receipt of the empty 
grid description by unloading the current grid description at 
step 206. 

Another possibility is that the name server 26 determines 
at step 192 that a URL corresponding to the detected position 
is not available. In this situation, the name server 26 sends 
an application location reply 208 to the electronic pen client 
22. The reply 208 may simply be empty to indicate that a URL 
is not available. Preferably, however, the reply 208 contains 
a grid exception defining the largest area possible around the 
detected position for which there is no corresponding URL. In 
response to the reply 208, the electronic pen client 22 sends 
a new grid reply 210 containing an empty grid description with 
a grid exception. Upon receiving the reply 210, the electronic 
pen 10 unloads the current grid description at step 212. 
Furthermore, assuming that the electronic pen 10 receives and 
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recognizes the grid exception information, the electronic pen 
10 may subsequently be able to determine that certain detected 
positions on the address pattern are not associated with any 
application without having to send a request to the name server 
26 or the application server 30. 

To implement the electronic pen 10, there should be some 
type of mechanism for allowing the electronic pen 10 to 
determine when to attempt to detect an address pattern, when to 
store detected positional information, and/or when' to forward 
or transmit the detected positions to the electronic pen client 
22. In addition, it is also desirable to allow a user of the 
electronic pen 10 to select between inking mode, wherein the 
electronic pen 10 writes ink or other markings on the paper or 
other surface with which the tip of the pen 10 comes into 
contact, and a non-inking mode, wherein the electronic pen 10 
does not leave ink marks when the tip of the electronic pen 10 
comes into contact with paper or other surfaces. 

In accordance with the present invention, it is possible 
to turn the inking function of the electronic pen 10 on and 
off. In particular, a user of the electronic pen 10 can use 
the pen 10 with an ink cartridge extended in a writing position 
or with the ink cartridge retracted so that the electronic pen 
10 does not leave ink markings. The ability to use the pen 10 
with its inking function turned on allows users to keep a paper 
copy of any drawings, handwritten text, or other markings made 
with their electronic pen 10, while the ability to use the pen 
10 in the non-inking mode facilitates use of the pen in 
connection with reusable or public data entry sheets or other 
public or reusable surfaces. 

In another aspect of the invention, the electronic pen 10 
can be used to navigate on a mobile phone 14 or other 
electronic device, such as a PDA, laptop computer, or personal 
computer, or to manipulate, enter, or delete data by writing on 
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a specially formatted paper. In particular, the electronic pen 
10 can relay the written information to the mobile phone 14 or 
other electronic device. This use of the electronic pen 10 and 
specially formatted paper permits most messaging and 
telecommunications control functions to be performed in a user 
friendly manner without the use of other devices. Moreover, 
actions resulting from the use of the electronic pen 10 and 
specially formatted paper can be represented on a display of 
the mobile phone 14 , PDA, or computer. 

For example, there is illustrated in FIGURE 11, a reusable 
calendar entry sheet 220. It is assumed that the calendar 
entry sheet 220 is printed on a paper that includes a portion 
of the address pattern (net shown) . Preferably, the address 
pattern is -either not visible to the human eye or is only 
faintly visible so that the address pattern does not present a 
significant distraction to users. The sensor in the electronic 
pen 10, however, can detect the address pattern, and therefore, 
the precise position of the pen 10 at any given time can be 
determined, even when the pen is in a non-inking mode. 
Accordingly, when the electronic pen 10 is used to fill in a 
date field 222, the motion of the electronic pen 10 over the 
address pattern can be detected. Then, based on an analysis of 
the strokes, the handwritten date can be stored and/or 
converted into text. Similarly, appointment information or 
other data can be written' in the appropriate fields of the 
calendar 224. By storing the detected positions, generating 
strokes based on the detected positions, and generating actions 
based on the strokes, the information written on the calendar 
can automatically be stored in a calendar application database 
(located, e.g., in a mobile phone 14 or in a remote server) 
associated with the user of the electronic pen 10. If desired, 
information can also be manipulated or deleted in a similar 
manner. Because the electronic pen 10 is preferably used in a 
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non-inking mode, the calendar entry sheet 220 can be reused. 
Furthermore, the same types of functions can be used to enter, 
delete, and manipulate data in an electronic phone book, task 
list, or scheduler. 

Referring now to FIGURE 12, there is illustrated an 
example of a reusable phone settings entry sheet 226. As with 
the previous example, it is assumed that the reusable phone 
settings entry sheet -226 is intended to be used when the 
electronic pen 10 is in a non-inking mode. Thus, by simply 
touching the electronic pen '10 to a desired setting in a ring 
volume field 228, the ring volume of a phone associated with, 
or otherwise selected by, the electronic pen 10, can be set. 
In addition, the electronic pen 10 can be used to select 
various other setting fields, such as a ring melody field 230, 
a greeting field 232, a phonebook entry field 234 for entering 
names and numbers into the phone internal memory, and fields 
relating to any other desired settings. When the electronic 
pen 10 is used to select or to enter information into these 
fields, such selections or entered information can be 
transmitted by the electronic pen 10 to the electronic pen 
client 22 contained within, for instance, a mobile phone 14 

Although the invention is described primarily in 
connection with a reusable calendar sheet 220 and a reusable 
phone setting entries sheet 226 in a non-inking mode, the 
invention is not limited to these applications. Instead, the 
electronic pen 10 can also be used for such data entry and 
storage when it is in an inking mode and can be used in 
connection with specially formatted paper to input any type of 
printed text or pattern in either the inking or non-inking 
mode. For example, the electronic pen 10 and an appropriately 
formatted paper or papers could also be used for navigating on 
a phone menu, dialing a number by writing it in a predefined 
"dialing" field, writing a message that is converted to a short 
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message service (SMS' message, telefax, or email message in the 
phone or in the electronic pen client 22, or saving notes in 
the phone 14 or in the electronic pen client 22 in the form of 
graphical or ASCII; text, characters, and/or pictures. 
Similarly, the preprinted paper can comprise a calendar, phone 
book, or other type of personal information manager 
application, or the preprinted paper can be formatted to 
facilitate entry of settings in a phone, personal computer, or 
other electronic device capable of communicating with 
electronic pen 10 . 

In accordance with another aspect of the present 
invention, the electronic pen 10 includes a paper touch sensor, 
or force sensitive detector, for determining when the 
electronic pen 10 is in contact with the paper. Preferably, 
the paper touch sensor is used to enable and disable the 
position sensing mechanism in the electronic pen 10 depending 
upon whether the electronic pen 10 is in contact with either a 
paper or other surface, regardless of whether the electronic 
pen 10 is in an inking or non-inking mode. For example, the 
paper touch sense.-. Cc.n be used to enable the position FIFO 
Block 38 shown in riGURE 7 when it senses that the electronic 
pen 10 is in contact with a paper or other surface and can be 
used to disable the position FIFO Block 38 when the paper touch 
sensor detects that the electronic pen 10 is not in contact 
with any surface. Accordingly, the electronic pen 10 will 
detect positions and generate strokes and actions only when the 
paper touch sensor detects that the electronic pen 10 is in 
contact with a paper or other surface. 

Referring now to FIGURES 13A and 13B, there is illustrated 
an electronic pen 10 having a paper touch sensor 236 for 
detecting whether the electronic pen 10 is in contact with a 
paper 240 or other surface regardless of whether an ink 
cartridge 238 is in an extended, or ink-on, position (FIGURE 
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13A) or in a retracted, or ink-off, position (FIGURE 13B) . 
Alternatively, instead of using an ink cartridge 238, the 
electronic pen 10 can utilize a pencil lead, a felt-tip marker, 
or any other type of marking tip for writing on paper or other 
surfaces . 

In yet another aspect of the present invention, the 
electronic pen 10 can be used to perform a mouse function. 
Accordingly, the electronic pen 10 can be used for navigation 
purposes (e.g., in a navigation field) when the paper touch 
sensor 236 detects that the electronic pen 10 is in contact 
with a paper 240 or other surface. In addition, a selection at 
a particular location can be indicated by "clicking" the tip of 
the electronic pen 10 against the paper 240 or other surface 
either by tapping the tip of the electronic pen 10 against the 
paper 240 or other surface or by pressing the tip of the 
electronic pen tip against the paper 240 or other surface with 
additional force (i.e., a force greater than a predetermined 
threshold force) . When using the electronic pen 10 to perform 
a mouse-like function, the ability to retract the ink cartridge 
238 provides an additional benefit in that the electronic pen 
10 is not writing and the ink cartridge 238 is not damaged when 
the user "clicks" the electronic pen 10. 

In accordance with another aspect of the present 
invention, an electronic reading device 10 can be used in 
connection with a predefined address pattern on a specially 
formatted surface 12 to activate a hyperlink. The hyperlink 
could function in a way similar to existing web page hyperlinks 
except that the hyperlink could be represented on any tyoe of 
addressed surface, electronic or non-electronic, rather than 
just on a display screen. In other words, the hyperlink could 
be printed on an addressed area of a paper, wherein a 
particular Internet address is associated with the address 
pattern contained in the addressed area. Thus, a textual or 
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graphical identification of the hyperlink (visible and 
understandable to a user) can be printed on an addressed area, 
which is preferably only slightly visible and is not easily 
decipherable by a user. 

In accordance with the invention, the hyperlink can 
initiate an execution of a function in the electronic reading 
device 10 or any other device. The types of functions can 
include a download or upload, to or from the electronic reading 
device 10 or other device, of text, images, executable 
software, audio, video, status information, control data, or 
position information. For example, by clicking on or touching 
the hyperlink with the electronic reading device, information 
could be retrieved from a remote server located at the Internet 
address and- downloaded to the electronic reading device 10 or 
15 to another electronic device, such as a mobile phone, that has 

an Internet browser (e.g., a web browser or a wireless 
application protocol (WAP) browser) . Similarly, an object or 
text printed on the addressed surface 12 can be associated with 
an electronic version of the object or text. Thus, touching 
20 the electronic reading device 10 to an image printed on the 

addressed surface 12 can retrieve a corresponding JPEG image, 
while touching -a text area retrieves an audio copy of the text 
(e.g., for use in a newspaper for the visually handicapped) . 
Alternatively, by clicking on or touching the hyperlink, 
25 information about the user of the electronic reading device 10 

could be sent to a server for storage or processing. 

Referring now to FIGURE 20, there is illustrated a 
schematic diagram of a hyperlink system 340 in accordance with 
a preferred embodiment of the present invention. When a user 
of the electronic reading device 10 touches or clicks on a 
hyperlink area 342 of a formatted surface 12 that includes a 
predefined address pattern, the electronic reading device 10 
detects the hyperlink address pattern 342. Execution of a 
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"click" can be done, for example, using optical or pressure 
sensitive detectors on the electronic reading device 10, using 
a click button on the electronic reading device 10, by simply 
touching the tip of the electronic pen 10 to a certain position 
or positions on the surface 12, by drawing a circle around a 
certain area on the surface 12 or by drawing a symbol on a 
certain area of the surface 12. The electronic reading device 
10 then sends the detected address to a mobile phone 14 via a 
Bluetooth™ interface 16, The mobile phone 14, in this case, 
acts as an electronic reading device client 22 and, as 
described above, uses the detected address to obtain a URL for 
a server associated with the detected address. Alternatively, 
the detected address can also be sent to other electronic 
devices, su'ch as a PC, PDA, or sent directly to a server. 
Moreover, instead of using a radio interface, other types of 
wireless technology, such as infrared light or inductive 
coupling, or wired technology, such as cable connections, can 
also be used. 

Using a WAP browser or web browser, the mobile phone 14 
then sends a message to a hyperlink server 348. The message 
indicates that the hyperlink 342 has been selected and 
preferably includes an electronic reading device identifier 
and/or mobile phone identifier so that the hyperlink server 34 8 
can identify the user. In particular, the message is sent over 
a radio interface 18 to a base station 20, which forwards the 
message via a general packet radio service network 344 and an 
IP network 346 to the hyperlink server 348. After receiving 
the message, the hyperlink server 348 can send a response to 
the mobile phone 14 and/or the electronic reading device 10. 
The response can include any type of information, such as a web 
page, that is associated with the hyperlink. Alternatively, 
the hyperlink server 348 can send information in the form of an 
email to an email address associated with the user of the 
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electronic reading device 10 and/or mobile phone 14. 
another alternative, the hyperlink server 348 can simply store 
an indication that the user has selected the hyperlink or can 
perform some type of processing of data received in the 
message . 

Such a hyperlink system 34 0 can be used for a wide variety 
of applications. In one embodiment of the invention, the 
hyperlink system 340 can be used to obtain more information 
about a product. The hyperlink address pattern 342 is included 
on a product, on product' packaging, or in a product 
advertisement. When the hyperlink address pattern 342 is 
detected by the electronic reading device 10, additional 
product information is retrieved via the Internet and displayed 
on the mobile phone display 350. 

In another embodiment of the invention, hyperlink address 
patterns 342 can be included in newspapers and books. 
Information could then be downloaded to an electronic device, 
in the form of text, still images, video, or audio, depending 
on the capabilities of the electronic device. For example,^ 
textual information could be sent to the mobile station 14 or 
a web-based version of a related article, or a link thereto, 
could be attached to an email delivered to the user's email 
address . 

In another alternative embodiment, a hyperlink address 
pattern 342 is printed on part of a business card or 
identification card. By detecting the hyperlink address 
pattern 342 with the electronic reading device 10, contact 
information (e.g., a vCard) or other personal information is 
downloaded to the mobile phone 14 or to a PDA. 

Another embodiment of the invention facilitates retrieving 
timetable information. By including a hyperlink address 
pattern 342 at a bus stop, a timetable can be retrieved and 
downloaded to the mobile phone 14 when the electronic reading 
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device 10 is scanned over or clicked on the hyperlink address 
pattern 342. Furthermore, by using a hyperlink address pattern 
342 that is unique for the particular bus stop, the hyperlink 
server 348 can identify the particular bus stop at which the 
user is located. Accurate information about when the next bus 
should arrive, including any delays, can then be sent to the 
mobile station 14 in the form of text or audio. Accordingly, 
the user could receive a voice message stating, for example, 
that "the bus will arrive at this stop in ten minutes.'' 

In another embodiment, 'the hyperlink address pattern can 
be used as an alternative to bar codes and magnetic strips. 
Thus, by scanning the hyperlink address pattern 342, the 
hyperlink server 348 can identify the entity or object with 
which the hyperlink address pattern 342 is associated. 

In yet another embodiment, a museum, zoo, amusement park, 
or other attraction can have a hyperlink address pattern 342 
placed in front of certain objects or areas (e.g., on signs 
associated with individual exhibits) . By detecting the 
hyperlink address pattern 342 with the electronic reading 
device 10, information or a link thereto can be sent in an 
email to the user's email address for reading at home. 

In another embodiment, different hyperlink address 
patterns 342 can be used to conveniently and quickly select and 
set office messages. For example, by using the electronic 
reading device 10 to select a state, such as u Out of office," 
vx Lunch," etc. , the user's phone mail and email settings could 
automatically be programmed with appropriate messages. In 
addition, visitors could use their own electronic reading 
device 10 to select a hyperlink address pattern 342 outside of 
the user's office to retrieve, to the visitor's mobile phone, 
a message containing the state currently selected by the user. 

In another embodiment of the invention, delivery persons 
can use an electronic reading device 10 on a hyperlink address 
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pattern 342 at delivery locations to verify that they are at 
the correct location. In addition, the hyperlink server 348 
could store a confirmation that the delivery person has 
actually been to a particular location along with the time of 
arrival . 

In yet another alternative embodiment of the invention, 
a hyperlink address pattern 342 can be used instead of normal 
text or magnetic strips on tickets. The hyperlink address 
pattern 342 could then be scanned to authenticate the ticket. 
Alternatively, both text and the hyperlink address pattern 342 
can be used so that the ticket can be both visually read by the 
user and electronically detected by an electronic reading 
device 10. 

Although various preferred embodiments of the method and 
apparatus of the present invention have been illustrated in the 
accompanying Drawings and described in the foregoing Detailed 
Description, it is understood that the invention is not limited 
to the embodiments disclosed, but is capable of numerous 
rearrangements, modifications, and substitutions without 
departing from the spirit of the invention as set forth and 
defined by the following claims. Furthermore, it shall be 
understood that the terms "comprises" and "comprising," when 
used in the foregoing Detailed Description and the following 
claims, specifies the presence of stated features, elements, 
steps, or components but does not preclude the presence or 
addition of one or more other features, elements, steps, 
components, or groups thereof. 
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WHAT IS CLAIMED IS: 

1. A system for electronic data entry, comprising: 
an electronic reading device, including: 

an optical detector for detecting a portion of 
a predefined address pattern on a specially formatted 
surface to the electronic reading device; and 

a transmitter for transmitting data relating to 
the detected portion of the predefined address pattern 
a separate electronic device for receiving the 
transmitted data and for performing a function corresponding to 
an area of the predefined address pattern that includes the 
detected portion of the address pattern. 

2. The system of claim 1, wherein the transmitter 
comprises a, cable. 

3. The system of claim 1, wherein the transmitter 
transmits information via a local wireless link. 

4. The system of claim 3, wherein the local wireless 
link comprises a Bluetooth radio interface. 

5. The system of claim 1, wherein the separate 
electronic device is selected from the group consisting of a 
mobile phone, a personal digital assistant, a television, and 
a personal computer. 

6. The system of claim 5, wherein the function comprises 
manipulating at least one setting for the separate electronic 
device . 

7. The system of claim 5, wherein the function comprises 
navigating an options menu for the separate electronic device. 

8. The system of claim 7, wherein a display on the 
separate electronic device displays at least a portion of the 
options menu. 



WO 01/61449 PCT/EP01/0139* 

9. The system of claim 1, wherein the optical detector 
detects a plurality of locations on the predefined address 
pattern corresponding to at least one image formed by the 
electronic reading c.e-ice on the specially formatted surface, 
the transmitter transmits data relating to the plurality of 
detected locations, and the separate electronic device receives 
the transmitted data and uses the received data in performing 
the function. 

10. The system of claim 9, wherein the at least one image 
comprises at least one handwritten character. 

11. The system of claim 9, wherein the separate 
electronic device comprises a mobile phone. 

12. The system of claim 11, wherein the mobile phone 
converts the received data into one of a short message service 
(SMS) message, a telefax, and an electronic mail. 

13. The system of claim 11, wherein the at least one 
image comprises a handwritten telephone number, the mobile 
phone operating to dial the telephone number. 

14. The system of claim 8, wherein the specially 
formatted surface comprises a calendar manipulation form, the 
separate electronic device converting the received data into 
data for entry in a calendar stored in the separate electronic 
device . 

15. The system of claim 8, wherein the specially 
formatted surface comprises an email form, the separate 
electronic device operating to convert the received data into 
an email for transmission by the separate electronic device. 

16. The system of claim 8, wherein the specially 
formatted surface comprises a phonebook manipulation form, the 
separate electronic device converting the received data into 
data for entry in a phonebook stored in the separate electronic 
device . 
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17. The system of claim 8, wherein the separate 
electronic device stores the received data in the form of a 
graphical image. 

18. The system of claim 8, wherein the separate 
5 electronic device converts the received data into text. 

19. The system of claim 8, wherein the specially 
formatted surface comprises a task list manipulation form, the 
separate electronic device converting the received data into 
data for entry in a task list stored in the separate electronic 

10 device. 

20. The system of claim 1, wherein the electronic reading 
device further includes a controller for sending the data 
relating to the detected portion of the predefined address 
pattern. 

15 21. The system of claim 1, wherein the separate 

electronic device comprises a server. 

22. The system of claim 21, wherein the function 
comprises a script corresponding to the detected portion of the 
address pattern. 

20 23. The system of claim 22, wherein the script is 

executed by the server. 

24. The system of claim 22, wherein the script is 

delivered to a second separate electronic device for execution 

by the second separate electronic device. 
25 25. The system of claim 21, wherein the function 

comprises a handwriting recognition application. 

26. The system of claim 25, wherein the function further 
comprises converting handwriting to text. 

27. The system of claim 21, wherein the function 
30 comprises a language translation application. 

28. The system of claim 21, wherein the function 
comprises a spelling help application. 
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29. The system of claim 21, wherein the function 
comprises creating an electronic credit card receipt. 

30. The system of claim 21, further comprising an 
electronic device having a display, wherein the function 

5 comprises sending information to the electronic device for 

displaying on the display. 

31. The system of claim 30, wherein the function is 
selected from the group consisting of displaying a web page on 
the display, displaying product information on the display, and 

10 displaying a translated word on the display. 

32. The system of claim 21, further comprising an 
electronic device selected from the group consisting of a 
mobile phone, a personal digital assistant (PDA) , a television, 
and a personal computer, wherein the function comprises sending 

!5 information to the electronic device, said information selected 

from the group consisting of an audio message for playback by 
a speaker on the electronic device and a textual message for 
display on a screen of the electronic device. 

33. The system of claim 21, wherein the specially 
20 formatted surface is preprinted with at least one icon, wherein 

a portion of the address pattern on which the at least one icon 
is preprinted is associated with the function. 

34. The system of claim 21, further comprising a notebook 
that includes a plurality of sheets of paper, each sheet of 

25 paper including an area of the address pattern associated with 

a particular application, wherein the server executes a 
particular application in response to a detection by the sensor 
of an area associated with the particular application. 

35. The system of claim 34, wherein the server monitors 
usage of the sheets of paper and sends a message prompting a 
user to obtain more paper based on said monitored usage. 

36. The system of claim 21, wherein the function 
comprises sending a message over the Internet. 
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37. The system of claim 36, wherein the message is 
selected from the group consisting of an email, a telefax, and 
a short message service (SMS) message. 

38. A method for data entry using an electronic reading 
device, comprising the steps of: 

detecting at least one position of an electronic 
reading device relative to a predefined address pattern on a 
specially formatted surface; 

transmitting data relating to the at least one 
detected position; 

receiving the transmitted data; and 

performing an operation using the received data, said 
operation corresponding to an area that contains the at least 
one detected position. 

39. The method of claim 38, wherein the operation 
comprises manipulating settings for a mobile device. 

40. The method of claim 38, wherein the operation 
comprises navigating in an options menu of a mobile phone. 

41. The method of claim 38, wherein the operation 
comprises entering data corresponding to the at least one 
detected position in an electronic calendar. 

42. The method of claim 38, wherein the operation 
comprises converting the at least one detected position into a 
short message service (SMS) message. 

43. The method of claim 38, wherein the at least one 
detected position corresponds to a handwritten telephone 
number, the operation comprising converting the at least one 
detected position into the telephone number. 

44. The method of claim 38, wherein the operation 
comprises converting the at least one detected position into an 
email message. 
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45. The methcd of claim 38, wherein the operation 
comprises entering data corresponding to the at least one 
detected position i l an electronic phone book. 

46. The methcd claim 38, wherein the at least one 
5 detected position corresponds to handwritten information, the 

•operation comprising saving the information in a memory. 

49. The method of cl aim 38, wherein the operation 
comprises entering data corresponding to the at least one 
detected position in an electronically stored task list. 
10 48. The method of claim 38, wherein the step of 

transmitting data relating to the at least one detected 
position comprises transmitting the data via a local wireless 
link. 

49. An electronic data entry system, comprising: 

15 a formatted paper having printed thereon a predefined 

address pattern; 

an electronic reading device for detecting portions 
of the predefined address patte rn and relaying information 
relating to the detected portions of the predefined address 
20 pattern; and 

a mobile phone for receiving the relayed information, 
wherein a user of the mobile phone can utilize the relayed 
information to execute a function within the mobile phone. 

50. A method of using an electronic reading device, 
25 comprising the steps of: 

detecting information written with an electronic 
reading device on formatted paper that includes a predefined 
address pattern; 

relaying the detected information to a mobile phone; 

30 and 

utilizing the relayed information to execute a 
function within the mobile phone. 
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51. An electronic reading device, comprising; 

a sensor for detecting at least a portion of a 
pattern on a specially formatted surface; 

a memory for storing a predefined function 
corresponding to the pattern; and 

a controller for performing the predefined function 
when the sensor detects at least a portion of the pattern. 

52. The electronic reading device of claim 51, wherein 
the memory stores a plurality of predefined functions, each 
predefined function corresponding to at least one specific 
pattern, and wherein the controller performs the corresponding 
predefined function when the sensor detects at least a portion 
of the at least one specific pattern. 

53. The electronic reading device of claim 51, wherein 
the sensor comprises a camera. 

54. The electronic reading device of claim 51, wherein 
the predefined function performed by the controller involves 
interacting with at least one other electronic device that 
further performs an additional function relating to the 
detected pattern. 

55. The electronic reading device of claim 51, wherein 
the predefined function comprises sending information to a 
separate electronic device. 

56. The electronic reading device of claim 55, wherein 
the predefined function is selected from the group consisting 
of sending information to an address on the Internet and 
sending information to one of a mobile phone, a personal 
digital assistant (PDA), a television, and a personal computer. 

57. The electronic reading device of claim 55, wherein 
the separate electronic device maps the information received 
from the controller to a particular server based on the 
detected pattern.- 
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58. The electronic reading device of claim 51, wherein 
the predefined function comprises storing information. 

59. The electronic reading device of claim 51, wherein 
the predefined function comprises deleting information from a 
memory . 

60. The electronic reading device of claim 51, wherein 
the predefined function includes retrieving information from a 
memory. 

61. The electronic reading device of claim 60, wherein 
the retrieved information is selected from the group consisting 
of predetermined text and a predetermined image. 

62. The electronic reading device of claim 51, wherein 
the predefined function comprises requesting that certain 
information be displayed on a display. 

15 63. The electronic reading device of claim 51, wherein 

the controller further compares the detected pattern with a 
pattern stored in the memory to determine whether the detected 
pattern matches the stored pattern. 

64. The electronic reading device of claim 51, wherein 

20 .the pattern comprises a portion of an overall address pattern 

formatted such that a substantially exact location on the 
overall address pattern can be identified based on an 
examination of a portion of the overall address pattern that is 
adjacent to the location. 

25 65. The electronic reading device of claim 51, wherein 

the pattern comprises a symbol printed on the specially 
formatted surface. 



54 



WO 01/61449 PCT/EP01/01398 

66. A method of using an electronic reading device, 
comprising the steps of: 

reading, with the electronic reading device, at least 
a portion of a specific pattern printed on a specially 
5 formatted paper; 

identifying a specific function associated with the 
specific pattern; and 

performing the specific function. 

67. The method of claim- 66, wherein the specific pattern 
10 comprises a symbol. 

68. The method of claim 66, wherein the specially 
formatted paper comprises a plurality of predefined areas, each 
associated with a predetermined function, one of the plurality 
of predefined areas including the specific pattern and 

15 associated with the specific function. 

69. The method of claim 68, further comprising the steps 

of: 

storing a first plurality of the predetermined 
functions in the electronic reading device; 
20 storing a second plurality of the predetermined 

functions in at least one separate electronic device; and 
contacting the at least one separate electronic 
device to perform the specific function when the first 
plurality of predetermined functions does net include the 
25 specific function associated with the specific pattern. 

70 . The method of claim 69, further comprising the step 
of retrieving an application description associated with the 
specific pattern. 

' 71 . The method of claim 66, wherein the specific function 
30 is selected from the group consisting of sending information to 

a separate electronic device, storing information, and deleting 
information . 
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72. The method of claim 66, wherein the specific pattern 
comprises an arrangement of dots on the specially formatted 
paper . 

73. The method of claim 66, wherein the step of reading 
5 at least a portion of ihe specific pattern involves reading a 

portion of the specific pattern substantially smaller than an 
area' covered by the specific pattern. 

74. The method of claim 66, wherein the specific pattern 
comprises a portion of an overall address pattern, different 

10 portions of the overall address pattern corresponding to 

different functions . 

75. The method of claim 66, wherein the step of 
identifying the specific function further comprises 
transmitting information relating to the read portion of the 

15 specific pattern from the electronic reading device to a 

server . 

76. The method of claim 66, wherein the step of 
performing the specific function comprises executing a script. 

77. The method of claim 66, wherein the function 
20 comprises a handwriting to text translation. 

78. The method of claim 66, wherein the function 
comprises a language translation application. 

79. The method of claim 66, wherein the function 
comprises a spell help application. 

25 80. The method of claim 66, wherein the function 

comprises sending information to a mobile electronic device. 

81. The method of claim 66, wherein the function 
comprises accessing a web page. 

82. The method of claim 66, wherein the function 
30 comprises creating an electronic credit card receipt. 

83. The method of claim 66, wherein the function 
comprises retrieving product information. 
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84. The method of claim 66, wherein the function 
comprises generating one of an email, a telefax, and a short 
message service (SMS) message. 

85. The method of claim 66, wherein the specific pattern 
is associated with an icon printed on a specially formatted 
paper . 

86. The method of claim 66, further comprising the step 
of identifying a sheet of paper having printed thereon the 
specific pattern, said identification performed using the read 
specific pattern, wherein the sheet of paper corresponds to at 
least one application executable on an electronic device. 

87. An electronic reading device, comprising: 

a sensor for detecting symbols as the symbols are 
written on»a surface with the electronic reading device; 

a memory for storing at least one predefined symbol; 

and 

a processor for comparing a symbol detected by the 
sensor with the at least one predefined symbol stored in the 
memory and, if there is a match, executing a function 
corresponding to the detected symbol. 

88. The electronic reading device of claim 87, wherein 
the sensor detects symbols by monitoring movement of the 
electronic reading device relative to an address pattern of the 
surface . 

89. The electronic reading device of claim 88, wherein 
the function executed by the processor further corresponds to 
the area of the address pattern on which the detected symbol is 
written. 
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90. A method of using an electronic reading device, 
comprising the steps of: 

detecting a symbol written with the electronic 
reading device on a specially formatted surface; 

, comparing the detected symbol with each of at least 
one stored symbol; and 

performing a function associated with one of the at 
least one stored symbol if the detected symbol matches said one 
of the at least one stored symbol. 

91. The method of claim 90, wherein the at least one 
stored symbol and the function associated with said one of the 
at least one stored symbol are stored in a server, further 
comprising the step of sending information relating to the 
detected symbol to the server. 

92. The method of claim 91, wherein the information 
relating to the detected symbol is sent using a web browsing 
protocol . 

93. The method of claim 91, further comprising the step 
of executing a script stored in the server in response to the 
detected symbol. 

94 . The method of claim 90, where the step of detecting 
the written, symbol comprises detecting a movement of the 
electronic reading device relative to an address pattern of the 
specially formatted surface. 

95. The method of claim 94, wherein the function 
associated with said one of the at least one stored symbol is 
further associated with an area of the address pattern on which 
the detected symbol is written. 

96. The method of claim 90 / wherein the function 
associated with said one of the at least one stored symbol is 
selected from the group consisting of retrieving predetermined 
text and retrieving a predetermined image. 
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97. An electronic reading system, comprising: 

a formatted surface having an area that includes a 
specific part of an address pattern, wherein a unique position 
on the address pattern can be identified from an examination of 
5 a portion of the address pattern; 

an electronic reading device including a sensor for 
detecting at least a portion of the specific part of the 
address pattern; and 

a server associated with the specific part of the 
10 address pattern for receiving a message from the electronic 

reading device, said message sent in response to said 
detection, and for performing a function in response to said 
message . 

93. The system of claim 97, wherein the address pattern 
15 comprises a pattern of dots. 

99. The system of claim 97, wherein said area of the 
formatted surface further includes printed hyperlink 
identification information. 

100. The system of claim 99, wherein the printed hyperlink 
20 identification information comprises a printed image, said 

function performed by the server comprising retrieving an 
electronic copy of the printed image. 

101 . The system of claim 99, wherein the printed hyperlink 
identification information comprises text, said function 
25 performed by* the server comprising retrieving an audio copy of 

the text. 

102. The system of claim 97, wherein the function 
comprises sending information associated with the specific part 
of the address pattern to an electronic device associated with 
a user of the electronic reading device. 
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103. The system of claim 97, wherein the function 
comprises sending inf jrraation associated with the specific part 
of the address pattern to an email address associated with a 
user of the electroni j reading device. 

104. The system of claim 97, wherein the electronic 
reading device further includes a radio transmitter for sending, 
data relating to the detected specific part of the address 
pattern to the server. 

105. The system of claim 104, further comprising a mobile 
station for receiving said data and for sending said message to 
the server. 

106. The system of claim 105, wherein the function 
comprises sending information associated with the specific part 
of the address pattern to the mobile station. 

107 . The system of claim 106, wherein the mobile station 
includes an Internet browser, said information comprising a web 
page. 

108. The system of claim 97, wherein the electronic 
reading device sends data relating to the detected specific 

. part of the address pattern to the server using one of an 
infrared data transmission, inductive coupling, and a cable 
connection . 

109. The system of claim 97, wherein the specific part of 
the address pattern relates to a product, the function 
comprising sending product information. 

110. The system of claim 97, wherein the formatted surface 
comprises a part of a printed publication. 

111. The system of claim 97, wherein the formatted surface 
comprises a business card. 

112. The system of claim 97, wherein the function 
comprises retrieving a transportation timetable. 



60 



WO 01/61449 PCT/EP01/01398 

113. The system of claim 97, wherein the specific part of 
the address pattern is associated with an object and the 
function comprises retrieving an identification of the object 
based on the specific part of the address pattern. 

114. The system of claim 113, wherein the server further 
sends an authentication of the object. 

115. The system of claim 97, wherein the specific part of 
the address pattern corresponds to an office message state 

116. The system of claim 115, wherein the function 
comprises activating one of* a phone mail message associated 
with the office message state and an email response message 
associated with the office message state. 

117. The system of claim 97, wherein the specific part of 
the address pattern corresponds to a geographical location 

118. The system of claim 117, wherein the function 
comprises storing an indication that the specific part of the 
address pattern has been detected by the electronic reading 
device . 

119. A method for selecting a hyperlink address, 
comprising the steps of: 

detecting a portion of an address pattern on a 
formatted surface of an object using an electronic reading 
device ; 

identifying an Internet address corresponding to the 
detected portion of the address pattern; 

sending a message to the identified Internet address; 

and 

performing an operation in a server for the 
identified Internet address in response to the message. 

120. The method of claim 119, wherein the operation 
comprises sending data to a client address associated with the 
electronic reading device. 
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121. The method of claim 120, wherein the client address 
identifies one of a user email address and a user electronic 
device . 

122. The method of claim 121, wherein the data comprises 
product information . 

123. The method of claim 121, wherein the object comprises, 
a printed publication. 

124. The method of claim 121, wherein the object comprises 
a business card. 

125. The method of claim 121, wherein the data comprises 
transportation schedule inf ormat ion . 

126. The method of claim 121, wherein the data comprises 
an identification of the object. 

127. The method of claim 121, wherein the data comprises 
information regarding an attraction. 

128. The method of claim 121, wherein the data comprises 
location information relating to a geographical location of the 
ob j ect . 

129. The method of claim 119, wherein the operation 
comprises authenticating the object. 

130. A user interface system, comprising: 

a specially formatted surface having printed thereon 
an address pattern; 

an electronic reading device, including: 

a sensor for detecting at least one position of 
the electronic reading device relative to a portion of the 
address pattern adjacent to the electronic reading device; 
and 

a controller for sending information relating 
to the at least one detected position; 

a server for receiving the information relating to 
the at least one detected position and executing a function 
corresponding to the at least one detected position. 
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